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LONG-TERM  GOAL 

The  long-term  goal  is  to  explore  and  test  new  techniques  that  can  validate  and  improve  the  applications 
of  the  Coupled  Ocean/ Atmosphere  Mesoscale  Prediction  System  (COAMPS)  high-resolution  nested 
grids. 

OBJECTIVES 

The  objectives  are:  compare  validation  results  on  COAMPS  coarse  and  find  grids;  investigate  methods 
and  types  of  observations  that  can  be  used  for  validation  for  high-resolution  mesoscale  model 
forecasts;  and  improve  the  flexibility  of  COAMPS  fine  grid  initialization  and  displacements. 

APPROACH 

The  traditional  statistical  metrics,  such  as  the  bias  and  root  mean  square  (RMS)  error,  provide  limited 
information  when  applied  to  high-resolution  model  validation.  High-resolution  model  forecasts  contain 
high  degree  of  variability  in  time  and  space  in  association  with  a  particular  type  of  weather  event. 
Averaging  statistics  over  a  long  period  of  time  from  a  variety  of  conditions  is  not  sufficient  to  describe 
the  spatial  and  temporal  variations  of  meteorological  events  derived  from  higher  resolution  model 
forecasts.  The  statistics  computed  from  a  composite  of  “look  alike”  events  may  provide  more  useful 
information  such  as  observed  and  model  simulated  climatology  for  that  event.  Specific  events  can  then 
be  precisely  defined  to  further  categorize  the  model  forecast.  Our  initial  approach  is  to  examine  a 
composite  of  wind-based  events  over  the  Mediterranean  Sea  using  the  Special  Sensor 
Microwave/Imager  (SSM/I)  satellite  observations  to  explore  the  type  of  validation  information  that  can 
be  extracted  in  an  effort  to  better  quantify  the  high-resolution  model  forecast  skill.  Other  COAMPS 
forecasted  weather  events  of  interest  to  the  US  Navy  can  then  be  validated  using  a  similar  compositing 
technique. 

The  COAMPS  validation  effort  also  needs  to  be  expanded  to  include  the  quantitative  precipitation 
forecast  (QPF).  The  standard  precipitation  validation  technique  requires  comparison  with  the  observed 
rain  analysis  usually  obtained  by  analyzing  high-density  rain  gauge  data.  However,  such  high- 
resolution  rain  data  is  difficult  to  obtain  globally.  With  advanced  satellite  technology,  an  experimental 
SSMI+  Tropical  Rainfall  Measurement  Mission  (TRMM)  satellite  retried  rain  rate  developed  by  NRL 
is  useful  for  COAMPS  QPF  validation,  especially  for  the  rain  data  void  ocean  area.  The  approach  is  to 
develop  rain  analysis  software  to  validate  the  satellite  retrieved  rain  rate  over  the  continental  US 
(CONUS)  region  against  the  high-density  rain  gauge  data  from  National  Center  for  Environmental 
Prediction  (NCEP)  on  a  common  grid.  Before  we  are  fully  confident  about  the  quality  of  the  satellite 
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retrieved  rain  analysis,  the  effort  will  be  concentrated  on  validating  COAMPS  QPF  over  the 
continental  US  where  NCEP’s  rain  gauge  data  is  easily  available. 

To  improve  flexibility  and  utility  of  the  high-resolution  forecasts,  the  approach  is  to  (a)  develop  a 
moving  nest  algorithm  to  follow  the  battleships  or  meteorological  phenomenon,  reducing  the  overall 
time  needed  to  generate  the  forecast  and  allowing  even  finer  grids  to  track  features  of  interest;  and  (b) 
perform  case  studies  and  sensitivity  tests  to  fine-tune  the  silhouette  terrain  parameters. 

WORK  COMPLETED 

1 .  Developed  software  to  use  the  NCEP  River  and  Forecast  Center  (RFC)  rain  gauge  data  for 
COAMPS  QPF  validation  over  CONUS.  Developed  COAMPS  rain  analysis  software  to 
analyze  the  rain  estimates  from  the  SSMI+TRMM  satellite  retried  rain  rate  and  surface 
METOC  rain  report  to  the  CO  AMPS  validation  grid.  Validated  the  SSMI+TRMM  rain  rate 
over  CONUS. 

2.  Computed  COAMPS  QPF  scores  over  the  CONUS  area  for  selected  periods. 

3.  Conducted  and  evaluated  the  event-based  verification  technique  using  the  SSM/I  wind 
observations  over  the  Europe  and  Eastern  Pacific  regions. 

4.  Implemented  and  tested  the  moving  grid  algorithm. 

5.  Improved  the  silhouette  terrain  algorithm  run  time  efficiency.  Conducted  case  studies  on  the 
effect  of  the  envelope  and  silhouette  terrain  analysis.  Performed  sensitivity  tests  on  the 
silhouette  parameters. 

RESULTS 

COAMPS  QPF  Validation:  A  summer  and  winter  month  rain  analysis  comparison  was  conducted  over 
the  CONUS  region.  The  NCEP  RFC  rain  gauge  analysis  was  used  to  validate  the  rain  analysis  with 
surface  METOC  station  data  and  the  satellite  retrieved  rain  rate.  The  results  showed:  (a)  the  analysis 
using  the  METOC  data  lacked  the  necessary  resolution  especially  in  the  mountainous  regions;  (b)  the 
areal  coverage  of  the  satellite  rain  analysis  was  comparable  to  the  NCEP  RFC  rain  analysis;  (c)  the 
satellite  rain  analysis  overestimated  the  rain  rate  over  the  mountain  and  coastal  areas  while 
underestimated  rain  rate  in  light  rain  areas.  In  general,  the  satellite  rain  analysis  has  problems 
estimating  the  precipitation  amount  accurately.  Although  the  retrieved  satellite  rain  rate  can’t  be  used 
to  validate  the  COAMPS  QPF,  it  can  be  used  as  an  indication  of  the  rain  area. 

The  COAMPS  QPF  scores  for  the  summer  month  of  July  1999  were  computed  using  eight  different 
thresholds  of  rain  rate.  The  QPF  scores  include  the  equitable  threat  score  (ETS)  and  the  bias  score.  The 
ETS  score  is  between  0.0  and  1.0  with  0.0  and  1.0  representing  no  skill  and  perfect  forecasts, 
respectively.  The  QPF  validations  were  performed  on  a  80  km  horizontal  grid  resolution. 

The  COAMPS  27  km  rain  forecasts  were  produced  with  two  different  versions  of  the  Kain-Fritch 
cumulus  scheme.  The  results  show  COAMPS  over-predicted  the  amount  of  light  precipitation  (<  10 
mm/day)  while  it  under-predicted  the  heavy  precipitation  (>  10  mm/day).  The  ETS  score  show  that 
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COAMPS  always  under-predicts  the  correct  precipitation  areas  in  all  rain  threshold  categories  with  the 
heavy  precipitation  threshold  (>  50  mm/day)  being  the  worst.  Comparison  of  COAMPS  QPF  vs.  the 
NCEP  ETA  and  Navy  NOGAPS  QPF  is  shown  in  Fig  1.  COAMPS  has  slightly  better  QPF  scores  than 
NOGAPS  but  worse  than  the  ETA  scores  (ETA  is  known  to  be  highly  tuned  for  QPF).  The  validation 
results  indicate  that  future  improvement  of  the  COAMPS  moist  physics,  including  convective  and 
explicit  schemes  for  the  forecast  of  heavy  summer-time  convective  precipitation,  is  needed. 


□  Nogaps 

□  Eta 

□  Coamps-1 

□  Coamps-3 


■  Nogaps 

□  Eta 

□  coamps  1 

□  coamps  3 


Figure  1.  COAMPS  equitable  thread  score  and  bias  score  for  July  1999. 

[Also  shown  are  the  ETA  and  NOGAPS  forecast  results.  The  validation  was  performed  on  a 
common  80  km  horizontal  grid  resolution.  Left:  equitable  thread  score ;  Right:  bias  score] 


Event-Based  Verification /Wind  events  with  wind  speed  greater  than  10  m  s'1  were  examined  over  two 
regions:  (a)  the  Mistral  over  the  Mediterranean,  and  (b)the  wind  surge  off  the  US  west  coast.  The  27 
km  grid  COAMPS  10  meter  wind  forecasts  were  validated  against  the  SSM/I  winds  retrieved  using  the 
Goodberlet  et  al.  (1990)  regression.  Contingency  statistics  (correct  hit,  missed  forecast,  false  alarm, 
and  correct  no)  were  calculated  based  on  the  overlap  between  the  wind  >  10  m  s'1  areas  predicted  by 
the  model  versus  those  in  the  observations. 

Results  for  the  21 -hour  COAMPS  Mediterranean  forecasts  between  15  and  28  November,  2000 
showed  the  probability  of  a  correct  forecast  of  wind  greater  than  10  m  s'1  is  76%.  Most  of  the  forecasts 
initialized  at  0000  UTC  contained  over  1000  SSM/I  observations;  between  200  and  400  observations 
met  or  exceeded  then  10  m  s'1  criteria.  The  majority  of  the  forecast  errors  were  due  to  an 
underestimation  in  the  aerial  coverage.  The  model  wind  speed  bias  with  respect  to  the  SSM/I  was  -0.9 
m  s'1,  reflecting  a  tendency  for  the  model  to  underestimate  wind  speeds.  The  average  daily  equitable 
threat  score  (ETS)  was  about  0.3.  When  ETS  was  recalculated  over  a  portion  of  the  Mediterranean 
where  most  of  the  ocean  grid  points  had  50-60  observations  over  the  two-month  period,  much  higher 
ETS  scores  were  obtained  (Fig  2).  Values  as  high  as  0.8  existed  south  of  the  French  coast  in  and  near 
the  oval  circle  shown  in  Figure  2.  This  region  is  well  known  for  frequent  strong  northerlies  (Mistral) 
that  result  from  the  channeling  between  the  Pyrenees  and  the  Alps  mountains.  Also  of  note  is  the 
region  of  low  scores  paralleling  the  coast,  which  result  from  coastal  contamination  of  the  SSM/I. 
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Figure  2  Equitable  threat  scores  calculated  at  individual  grid  points. 

[All  ETS  values  have  been  multiplied  by  ten.  An  area  of  elevated  scores  likely  associated 

with  the  Mistral  is  indicated  by  the  oval.] 
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Figure  3  Contingency  statistics  for  the  42  hour  forecast.  COAMPS  10  meter  wind  occurrences 
greater  than  10  m/s  (Nov-Dec,  2000)  compared  with  the  SSM/I  observations.  The  colored  plots 
represent  (a)  probability  of  correct  forecast  and  (b)  probability  of  the  false  alarm  respectively. 


The  contingency  statistic  in  the  region  off  the  US  west  coast  from  November  to  December,  1999 
showed  an  opposite  result.  In  this  region  COAMPS  produced  a  greater  number  of  strong  wind 
events  than  observed.  A  high  probability  appeared  in  the  northern  portion  of  the  validation  region 
(Fig.  3a)  where  a  higher  occurrence  of  observed  strong  wind  events  existed.  Examination  of  the 
false  alarm  probability  (Fig.  3b)  shown  over  the  southern  region  indicates  that  COAMPS  had  a 
higher  probability  to  forecast  strong  wind  events  not  observed  by  the  SSM/I.  This  result  may 
indicate  a  possible  phase  error  in  the  southern  portion.  Further  examination  is  required  to  determine 
the  nature  and  the  magnitude  of  this  phase  error  if  it  indeed  exists  in  the  COAMPS  forecasts. 


4 


Moving  Grid:  Several  48  hour  COAMPS  simulations  for  hurricane  Gordon  (00Z  17-19  September, 
2000)  were  performed  to  examine  the  COAMPS  moving  nest  algorithm.  The  moving  nest  domain  size 
was  about  five  times  smaller  than  the  fixed  nest,  and  was  moved  every  6  hours  to  a  prescribed  location 
based  on  the  position  of  hurricane  Gordon  obtained  from  the  control  simulation  (fixed  nest).  The 
results  showed  the  moving  nest  simulation  was  able  to  reproduce  nearly  identical  hurricane  features 
compared  to  the  fixed  nest  case.  A  threefold  cpu  time  speed  up  was  obtained  using  the  moving  nest 
option. 

Improved  Silhouette  Terrain:  The  two  weeks  averaged  COAMPS  27  km  statistics  (30  September  to  14 
October,  1998)  over  the  Alps  showed  a  slightly  improvement  in  the  bias  and  RMS  scores  at  levels 
below  750  mb.  The  original  terrain  created  more  precipitation  on  the  mountain  peaks  than  the 
silhouette  terrain  due  to  a  higher  topography  value  at  the  peak.  Several  27  km  model  simulations  (5-7 
April,  2001)  were  perfonned  over  the  CONUS  region  where  the  observed  NCEP  RFC  rain  analysis  is 
available  to  examine  this  precipitation  difference.  When  compared  to  the  observed  precipitation,  the 
silhouette  runs  were  closer  to  the  observed  precipitation  amount  and  pattern  even  at  areas  downstream 
from  the  mountain.  The  tests  on  the  several  silhouette  parameters  showed  small  sensitivity  to  the 
averaged  bias  and  RMS  scores.  The  total  averaged  precipitation  also  showed  negligible  small 
differences. 

IMPACT/APPLICATIONS 

The  COAMPS  QPF  validation  over  the  CONUS  area  showed  the  COAMPS  moist  processes  (explicit 
and  convective)  were  not  able  to  produce  enough  precipitation,  especially  in  the  heavy  precipitation 
category.  These  results  provide  a  benchmark  for  future  improvement  of  COAMPS  QPF.  The  software 
that  was  developed  to  perform  rain  analysis  and  QPF  validation  will  help  to  continually  validate  the 
improvement  of  COAMPS  moist  processes. 

The  initial  implementation  of  the  moving  nest  algorithm  demonstrated  the  option  to  follow  a  battle 
group  (moved  by  locations).  An  automatic  moving  nest  following  a  hurricane  is  under  development  in 
collaboration  with  6.2  ONR  program  for  the  Improvements  to  Tropical  Cyclone  Model  Forecasts.  The 
automatic  moving  algorithm  will  allow  a  higher  model  grid  resolution  forecast  to  better  track  the 
hurricane  movement  and  intensity. 

TRANSITIONS 

Improved  algorithms  for  the  silhouette  terrain  and  moving  nests  will  transition  to  6.4  programs  (PE 
060243 5N  and  PE  0603207N)  for  applications  within  COAMPS  and  for  subsequent  transition  to  Fleet 
Numerical  Meteorology  and  Oceanography  Center  (FNMOC)  and  Regional  Naval  Meteorology  and 
Oceanography  Centers  for  operational  use. 

RELATED  PROJECTS 

Related  6.2  projects  within  PE  0602435N  are  BE-35-2-18,  for  the  Mesoscale  Modeling  of  the 
Atmosphere  and  Aerosols,  BE-35-2-44,  for  the  Advanced  Moist  Physics  Modeling,  and  BE-35-2-19, 
for  the  Exploratory  Data  Assimilation  Methods. 
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